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Abstract 

The application of telerobotics to aircraft de- 
pot maintenance and remanufacturing is described 
and a telerobotics architecture for the application 
is discussed . Telerobotics will enhance process 

quality and could potentially decrease turn-around 
time and costs while moving human operators from 
hazardous work areas to safe and comfortable op- 
erator control stations. The approach is to aug- 
ment, not replace y the human operator by blending 
human skills with robotics capabilities. Configura- 
tions of the architecture for telecrane } mobile car- 
rier , and gantry applications are shown. 

1. Introduction 

This paper summarizes a study performed by 
the Jet Propulsion Laboratory for the Air Force 
Material Command (AFMC) Robotics and Au- 
tomation Center of Excellence (RACE) to eval- 
uate the feasibility of telerobotic solutions to C- 
5A heavy lifter aircraft maintenance processes and 
develop a telerobotics architecture for the appli- 
cation [1]. The architecture was developed for 
general depot maintenance and remanufacturing 
applications and applied to the C-5 A application. 
Several implementation options suitable for inser- 
tion into a variety of depot applications that sup- 
port the C-5 A heavy airlifter are described. 


The Aircraft Directorate at the San Anto- 
nio Air Logistics Center (SA-ALC) is responsi- 
ble for depot level maintenance on the C-5 air- 
frame. The efficiency and productivity of many 
of the required repair processes will benefit from 
the insertion of telerobotics technologies. Small 
batch size, feature uncertainty, and varying work- 
loads make hard automation impractical for a wide 
range of depot processes. Systems are needed that 
can bridge the gap between manual control and 
complete automation. Supervised autonomy and 
shared control technologies provide intermediate 
solutions where the human and machine collabo- 
rate to perform tasks. In supervised autonomy, 
robotic tasks are interactively developed by the 
operator and then executed autonomously [2]. In 
shared control, control inputs during task execu- 
tion are provided both by an operator, e.g., using a 
hand controller, and an autonomous system [3]. A 
more complete description of telerobotics systems 
can be found in [4]. The goal is to augment, not 
replace, the human operator by blending human 
skills with robotics capabilities. 

Aircraft depot maintenance and remanufac- 
turing provides a wide range of challenges for 
robotics. The physical scale of the applications 
includes stripping paint from a C-5 heavy lifter to 
remanufacturing small individual parts. The parts 
generally arrive individually or in small batches 
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and a wide variety of parts are remanufactured. 
Due to the wide variety and scale of the appli- 
cations, the maintenance and remanufacturing is 
now done almost exclusively manually. Example 
depot applications which the architecture must ap- 
ply to include: painting of the C-5A exterior in 
a dedicated hanger facility; painting of removed 
piece parts in a robotic workcell; stripping of paint 
from the C-5A exterior in a dedicated hanger facil- 
ity; surface finishing in form of removing material 
from patches and polishing metal to a high gloss 
finish in a robotic workcell; Surface cleaning of re- 
moved parts in a robotic workcell through appli- 
cation of bicarbonate of soda particulate stream; 
and sealing and desealing of aircraft fuel tanks. 

It is expected that telerobotics can provide 
many benefits to aircraft depot maintenance and 
remanufacture. Limited manpower resources limit 
the number of aircraft that can be remanufac- 
tured. Telerobotics can augment the productiv- 
ity of operators allowing a greater rate of aircraft 
throughput. In many instances telerobotics can 
provide better process control, e.g., paint can po- 
tentially be sprayed on an aircraft more uniformly 
than by an operator leading to reduced average 
thickness and cost savings in paint and aircraft 
weight. There are various hazardous work situa- 
tions and environments in aircraft depot mainte- 
nance and remanufacturing areas including: chem- 
ical contaminants in the air and on shop surfaces; 
handling large, bulky support equipment; exces- 
sive vibration, especially of hand-operated equip- 
ment; and excessive atmospheric heat and humid- 
ity (up to 100 deg.F, 95% humidity). Telerobotics 
allows placing a manipulator in the hazardous en- 
vironment and moving the operator to a safe and 
comfortable operator control station. Addition- 
ally, there are tedious applications which cause fa- 
tigue and subsequent errors, e.g., paint stripping 
and deriveting. Many of these tasks can be accom- 
plished with the operator supervising a telerobotic 
system to perform the task resulting in greater ef- 
ficiency and quality. 

Since the telerobotic architecture was de- 
signed for use across a wide variety of depot air- 


craft and maintenance and remanufacturing appli- 
cations, there are a large number of requirements 
it must satisfy. The architecture must accommo- 
date different types of robotic manipulators with 
varying degrees of freedom with modular changes 
only to interface code. It must accommodate dif- 
ferent types of transport and positioning devices 
for robots and piece parts with modular changes 
only to interface code. Initialization and monitor- 
ing must be automated and rapid. Human oper- 
ations shall be able to safely operate within the 
range of motion of most manipulating and posi- 
tioning devices through built-in safety protocols 
(hardware, software, and/or procedural) Smooth 
transitions to manual workaround modes must be 
possible during automation downtimes for main- 
tenance, upgrades, etc. The architecture must 
accommodate different, unmodeled parts in all 
piece part applications. Software and hardware 
upgrades shall cause minimal down time. 

2. Example Application: C-5A Aircraft 
Maintenance and Remanufacturing 

The remanufacturing processes that support 
depot level maintenance of C-5 aircraft are repre- 
sentative of a wide variety of dual-use applications. 
Applications include stripping the external surface 
paint and then repainting, painting removed parts 
in a robotic workcell, skin repair, surface cleaning 
of removed parts through application of bicarbon- 
ate of soda particulate stream, surface finishing for 
patches, and polishing metal surfaces. A unique 
aspect of working on large airframes (the C-5A is 
over 247 ft. long and with a wingspan over 222 ft) 
is the requirement for large positioning systems. 
Several alternatives are possible. The first option 
is the telecrane concept where a special facility 
provides telecranes upon which the manipulators 
are mounted, as shown in figure 1. Such a tele- 
crane facility is presently used at Kelly AFB which 
positions human operators around the aircraft for 
servicing applications (paint stripping with plas- 
tic beads). The telecranes do not have positioning 
sensors so either positioning sensors would have 
to be added, or some other method would have to 
be used to determine the position of a manipula- 


631 



tor mounted on a telecrane. A second approach 
is to use mobile carriers where manipulators are 
mounted on mobile bases and the mobile bases 
are capable of being positioned around the air- 
craft, as shown in figure 2. Another option is to 
use an overhead gantry system where manipula- 
tors are mounted on mobile gantries. These trans- 
port methods apply to tasks which are done on 
the physical aircraft structure, such as painting, 
but there are also many tasks which are done on 
piece parts in separate workcell rooms such as re- 
pair and painting. 

3. Telerobot Architecture 

A telerobot architecture was developed to 
provide a near-term solution for implementation 
of a telerobotics system for C-5A servicing. Var- 
ious architectures were evaluated such as the 
DOE GIST architect ure[5], the NIST NASREM 
architecture[6], and the NASA/JPL local-remote 
architecture[2]. The architecture developed here 
has ideas common to all of these architectures. 
The GISC architecture provides the important 
concept of intelligent subsystems. The NASREM 
architecture provides valuable contributions in the 
coordination of task decomposition, modeling, and 
sensing. The NASA/JPL architecture provides 
the valuable concept of independent data-driven 
software modules to collectively provide general 
task execution capability. 

The architecture developed for aircraft main- 
tenance and remanufacturing is shown in figure 3. 
The architecture is nominally separated into lo- 
cal and remote sites corresponding to the location 
of the operator and robotic systems, respectively. 
The actual computing resources can be physically 
located near the operator, robotic system, or sep- 
arate from either. The primary constraint is that 
sufficient communication bandwidth is provided. 
The basic concept of the near-term system is that 
there exist subsystems which have sufficient inher- 
ent capability to execute a wide range of task types 
either independently or in coordination with other 
subsystems. A task program is generated by the 
local site which describes the task to be executed 


either by an independent subsystem or through 
coordination of subsystems. The task program 
can be executed in various ways depending on the 
level of capability of the coordinating subsystems. 
The desired solution is to allow distributed au- 
tonomous control of the coordinating subsystems 
by separating the task program into subsystem 
task programs. The subsystem task programs can 
then be executed by a task program sequencer, 
possibly at the local site operator control station, 
or sent to the subsystem controller for execution 
within the subsystem controller, if possible. Sub- 
system inherent capabilities are programmed off- 
line so that during task setup and execution the 
subsystems already have the necessary inherent 
task execution capabilities. 

Various maintenance and remanufacturing 
scenarios provide a poorly structured environment 
so that sensing the environment is necessary to 
generate or update a model of the environment. 
For example, neither the manipulators nor the air- 
craft will be positioned accurately to a well known 
location a priori to task execution. Before, or dur- 
ing, task execution, the relative positions of the 
manipulator and aircraft area of interest must be 
determined. A main object knowledgebase is pro- 
vided which stores global state information. Each 
subsystem also has its own database which in- 
cludes relevant information from the object knowl- 
edgebase and information generated from sensing 
the environment during task execution. The ob- 
ject knowledgebase and subsystem database are 
kept consistent for common information. Envi- 
ronment modeling can be done in various ways. 
Autonomous subsystem tasks can include, or have 
primarily, modeling elements. Alternatively, the 
operator can interact with the system to aid in 
developing models of the environment. For appli- 
cations which require highly accurate positioning, 
such as deriveting, it is likely that either sensor 
based position servoing or shared control will be 
necessary. An a priori generated model of the rivet 
pattern is unlikely to have the accuracy relative 
to the real rivet pattern that would be necessary 
for rivet removal. Sensor based position servoing 
would likely utilize real-time vision with an arm- 
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Figure 3: Telerobot architecture for aircraft maintenance and remanufacturing 




















mounted camera. A proximity sensor and possibly 
a sensor to measure surface tilt might also be used 
concurrently to control the position of the manip- 
ulator relative to the target. Rivets are difficult to 
find autonomously since the rivets have approxi- 
mately the same color as the skin. Also, for pre- 
viously repaired skin sections, the rivet pattern 
may not be known a priori. Therefore, for rivet 
removal, the operator can facilitate the use of the 
automated vision and sensing system by designat- 
ing the rough location of the rivets to be removed. 
A video image of the skin section is provided on 
a monitor for the operator. If a model of the skin 
section is available, then it is overlayed on top of 
the video image (a ghosted image or perhaps wire 
frame). Otherwise an approximate model of the 
skin is generated to provide a three dimensional 
surface upon which to designate rivet locations. 
The operator then utilizes an input device to move 
a cursor to the rivet locations seen on the video im- 
age and selects the rivets to store their locations 
in the object knowledgebase. These approximate 
locations can then be used as starting locations for 
the automatic sensor based localization later. It is 
often useful in the task programs to specify ob- 
jects and locations symbolically rather than with 
absolute locations. Then the task program can be 
generated independently of the actual locations. 
The actual locations of objects can be generated 
later either independently and stored in the ob- 
ject knowledgebase or as part of the task where 
operator input is automatically requested. Shared 
control can also be used to specify destinations. 
Here the operator uses an input device such as a 
six DOF hand controller to position the manipu- 
lator above the rivet. The proximity and/or tilt 
servoing could be occurring simultaneously to con- 
trol the distance to the surface, depending on the 
method for removing the rivets. In this case the 
operator replaces the vision system. 

It is desired that task description be as sim- 
ple as possible for the operator. Therefore, as 
much intelligence as possible is designed and pro- 
grammed into the system. For a sophisticated 
implementation, the operator would provide high 
level goal based information and the system would 


autonomously generate the associated task pro- 
grams. A more realistic near-term system would 
require greater interaction with the operator to 
develop a new task. It is desired that the opera- 
tor interact with the system primarily within the 
video/graphical environment, i.e., in a telepres- 
ence sense, both for task description and task ex- 
ecution. For task description, the operator would 
move the graphical manipulators via an input de- 
vice such as a six DOF hand controller. The ob- 
jects to interact with could be selected directly, or 
implied by proximity or context. The tool which 
the manipulator is carrying, along with the pre- 
vious task steps and the selected object, provide 
a large amount of context information which the 
system could use to automatically suggest to the 
operator, or select, the next action to take[2]. The 
actions could be the subtask segments from the 
task knowledgebase. 

The remote site subsystems will vary in the 
types of systems which they will control, in capa- 
bility, and in vendor source. For some subsystems 
the task program will have to be translated into its 
command language. For other subsystems, a task 
program might be used directly. There are several 
types of control and coordination which may be 
needed within subsystem control and between sub- 
systems. Closed loop control implies that there is 
a close coupling between sensory data and control 
commands to the devices. One subsystem provides 
cooperative control of its associated devices. Mul- 
tiple subsystems can be coordinated to achieve a 
task goal. 

Configurations of the architecture shown in 
figure 3 for telecrane, mobile carrier, and gantry 
applications are shown in figures 4- 6. 

4. Evolution of the Telerobotic 
Architecture 

The architecture shown in figure 3 supports 
near-term system development and evolutionary 
growth. Most of its basic features can be provided 
by existing vendors of automation and robotics 
technology. One drawback of current technology 
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Figure 5: Focused Mobile Carrier Control Architecture 
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Figure 6: Focused Gantry Control Architecture 


























is that it is difficult to integrate systems from dif- 
ferent vendors when a significant amount of con- 
trol and modeling information is passed between 
layers in the architecture, since this information 
is often stored in different formats. The evolu- 
tionary direction of the architecture is to provide 
subsystems with increasing levels of intelligence 
which can be provided with goal based informa- 
tion rather than control based information which 
is prevalent with current technology. The intel- 
ligent subsystems would then autonomously re- 
quest resources from other parts of the system such 
as the object knowledgebase. The resulting task 
programs would then be significantly smaller and 
quicker to generate. Protocols for communicating 
requests and information between the subsystems 
need to be developed. This approach is consistent 
with the goals of the Next Generation Controller 
program [7] which is developing a similar architec- 
ture for machine tool control. In the next year this 
effort will work more closely with the NGC effort 
to attempt to develop common interfaces and a 
common evolutionary architecture. The operator 
remains an integral part of the evolutionary in- 
telligent architecture. In such an architecture the 
operator could become one subsystem with mul- 
tiple capabilities or could be modeled as multiple 
subsystems. Also, the operator could act as one 
part of one of the subsystems such as the case de- 
scribed above where the operator performed the 
visual servoing for rivet localization. The system 
would then request input from the operator for 
information it cannot generate automatically, just 
as it would query one of the other parts of the 
system. 

5. Conclusions 

Application of telerobotics to aircraft depot 
maintenance and remanufacturing was discussed. 
The requirement to reduce technology insertion 
and system life cycle costs mandated the design of 
a generic architecture which can be implemented 
in the near-term and and still provide an evolu- 
tionary growth path. Most of the basic features 
of the near-term architecture are available from 
existing vendors. The evolutionary architecture 


utilizes increasing intelligence in the various mod- 
ules of the system resulting in a more distributed 
autonomous control system. A commercialization 
study is underway. 
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